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Abstract 

Test  and  evaluation  of  modern  weapon 
systems  is  a  complex,  costly,  and  time- 
consuming  process.  Weapon  System  evaluators 
are  presented  with  many,  often  times  conflicting 
issues.  For  example,  to  demonstrate  system 
performance  at  the  required  confidence  values 
often  requires  that  an  excessive  number  of  flight 
tests  be  performed.  Unfortunately,  in  today’s 
funding  constrained  environment,  this  is  not 
practical  from  a  cost  standpoint.  For  this  reason, 
the  use  of  computer  simulations  in  the  Test  and 
Evaluation  community  has  emerged  as  a 
primary  means  of  verifying  over-all  weapon 
system  performance.  While  models  and 
simulations  have  their  roles  in  performing  high 
numbers  of  Monte  Carlo  runs,  they  usually 
require  a  high  level  of  verification  and  validation 
to  ensure  proper  representation  of  the  system, 
threat  and  test  environment.  The  concept  of 
virtual  flight-testing  provides  an  excellent 
compromise  between  performing  actual  flight 
tests  and  running  traditional  digital  simulations. 
Virtual  flight-testing  utilizes  actual  flight  test  data 
taken  on  multiple  object  types,  from  numerous 
tests  to  create  new  flight  test  scenarios  that 
were  never  physically  flown.  The  ability  to 
manipulate  the  flight  test  data  in  both  range 
(space)  and  time  allows  existing  flight  test  data 
(often  from  multiple  tests)  to  be  combined  to 
create  a  new  “virtual”  flight  test.  This  process 
also  provides  a  much  higher  level  of  confidence 
than  traditional  simulations,  since  all  the  data  is 
real.  All  the  signatures  of  the  test  vehicles  are 
real,  so  the  threat  model  validity  is  no  longer  an 
issue.  Similarly,  all  the  sensor  effects  are 
inherently  included  in  the  data,  so  the  sensor 
model  validity  is  no  longer  an  issue  either. 

Virtual  flight-testing  is  a  proven  capability,  which 
has  evolved  and  merged  out  of  the  data  analysis 
and  simulation  areas.  Signature  data,  taken  at 
appropriate  aspect  angles,  can  be  “re-flown”  on 
any  three  degree-of-freedom  trajectory  desired. 
This  signature  data  may  come  in  the  form  of 
flight  data  or  static  pattern  data  taken  from  a 
range.  The  only  restriction  associated  with  the 
flight  test  data,  is  that  only  trajectories  with 


“observed”  aspect  angles  may  be  flown.  In  other 
words,  an  outbound  trajectory  with  high  aspect 
angles  (90-180°)  cannot  be  created  from  an  in¬ 
bound  flight  test  with  low  aspect  angles  (0-90°). 

As  long  as  this  restriction  is  not  violated,  any 
observed  flight  test  target  can  be  re-flown  on  any 
desired  trajectory.  This  is  an  extremely  powerful 
capability,  which  has  been  demonstrated  on 
multiple  projects  for  the  U.S.  Army  Space  and 
Missile  Defense  Command. 

Introduction 

As  noted  in  the  abstract,  virtual  flight  testing 
is  a  powerful  concept  that  can  preserve  valuable, 
limited  test  resources  while  simultaneously 
providing  high  statistical  confidences.  The 
concept  is  relatively  simple:  use  existing  flight  test 
data  in  an  “artificial  environmenf  to  create  new 
and  distinctly  different  flight  tests  which  were 
never  actually  flown.  On  the  surface  this  may 
sound  far-fetched  or  impossible,  but  the  concept 
has  been  demonstrated  on  several  different 
occasions  and  an  example  will  be  shown  in  this 
paper.  This  paper  will  outline  the  virtual  flight 
testing  concept  and  provide  the  basic  process 
used  to  create  new  scenarios.  It  will  also  highlight 
the  uses,  benefits  and  the  limitations  associated 
with  virtual  flight  testing.  Lastly,  a  host  of  possible 
scenarios  and  applications  will  be  identified  which 
might  intrigue  the  reader  to  learn  more  about  what 
is  being  done  in  this  fascinating  area  which 
marries  flight  test  data  and  simulations  together. 

Background  Of  Virtual  Flight  Testing 

XonTech,  Inc.  has  historically  been  involved 
in  three  main  business  areas  related  to  missile 
defense.  These  three  focuses  have  largely, 
although  not  exclusively,  focused  on  radar 
applications  in  testing  and  data  analysis,  algorithm 
development  and  simulations.  This  unique 
background  made  virtual  flight  testing  a  natural 
evolution  for  XonTech  as  a  company.  One  of  the 
primary  needs  of  the  algorithm  developers  has 
always  been  live  data  to  test  the  algorithms 
against.  Additionally,  one  of  the  primary  needs  of 
the  simulation  designers  is  to  verify  the  simulation 
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against  live  sensor  and  threat  data.  Having  a 
wealth  of  this  type  of  data  has  made  algorithm 
and  simulation  testing  and  verification  possible. 
Unfortunately,  even  with  this  extensive  database 
of  live  flight  test  data,  even  XonTech  found  a 
lack  of  relevant  live  data  to  test  some  algorithms 
and  simulation  functionality.  A  very  good 
example  of  this  is  in  the  area  of  kinetic  kill 
intercepts  of  Theater  Ballistic  Missiles  (TBMs). 
There  is  not  much  (if  any)  relevant  flight  test 
data  to  test  Ballistic  Missile  kill  assessment 
algorithms  against.  Additionally,  even  when  a 
test  set  of  relevant  data  did  exist,  the  analysts 
would  typically  want  more.  This  has  especially 
been  true  where  algorithm  verification  is 
concerned  because  many  of  the  algorithms  have 
extremely  stressing  statistical  performance 
requirements,  which  are  being  verified.  To 
meet  these  requirements  might  require 
thousands  of  data  sets,  which  obviously  are  not 
available. 

The  solution  to  this  problem  has  been  to 
artificially  manipulate  the  test  data  into  “new” 
scenarios.  In  the  case  of  a  kinetic  intercept  of  a 
TBM,  for  example,  it  will  be  shown  how  test  data 
from  numerous  sources  can  be  combined  to 
create  a  virtual  intercept.  Additionally,  the 
characteristics  of  the  intercept  event  can  be 
modified  to  meet  whatever  conditions  the  user 
desires.  For  example,  the  specific  numbers  of 
pieces  of  debris  can  be  specified.  The  intercept 
time  can  be  moved  forward  or  backward  in  time. 
The  intercept  point  can  be  changed.  The  debris 
velocity  can  be  changed.  The  width  of  the 
debris  cloud  as  a  function  of  time  can  be 
changed.  A  surviving  warhead  or  other  large 
objects  can  be  embedded  in  the  debris.  All  of 
these  important  parameters  can  be  modified  or 
manipulated  to  assess  their  impact  on  the  kill 
assessment  algorithm’s  performance. 

Why  Do  Virtual  Flight  Testing? 

Conventional  flight  testing  of  Ballistic 
Missile  Defense  systems  has  become  extremely 
complex  and  costly.  Numerous  programs  have 
had  the  number  of  planned  flight  tests  drop 
significantly  due  to  cost.  This  was  seen  on  the 
THAAD  Program,  which  was  originally  planned 
for  twenty  flight  tests  at  the  time  it  went  through 
the  Milestone  I  Defense  Acquisition  Board.  The 
National  Missile  Defense  Program  has  already 
limited  the  number  of  flight  tests  to  less  than  ten, 
due  to  the  costs  and  complexities.  In  an  effort  to 
save  time  and  dollars,  virtual  flight  testing  may 


be  viewed  as  a  reasonable  compromise  between 
conventional  flight  testing  and  traditional  digital 
simulation. 

One  of  the  problems  with  traditional 
simulations  is  verification,  validation  and 
accreditation  of  the  simulation.  This  must  be 
examined  on  several  levels.  First,  the  model  must 
adequately  represent  the  sensor  or  system  it  is 
modeling  in  enough  fidelity  to  assess  the 
predicted  performance.  Secondly,  the  targets  and 
threat  must  be  modeled  in  sufficient  detail  to 
ensure  that  the  model  is  properly  considering  all 
of  the  target  characteristics  and  attributes  which 
might  affect  performance.  Lastly,  the 
environments  must  be  modeled  in  enough  detail 
to  represent  “the  real  world”  effects  that  will  be 
seen  in  a  tactical  environment. 

Virtual  Flight  Testing  does  not  have  these 
three  concerns,  because  the  data  is  real.  An 
actual  sensor  is  used  to  collect  the  data,  so  all  the 
relevant  sensor  effects  (for  that  particular  sensor 
and  sensors  which  are  similar)  are  inherently 
included  in  the  data.  Likewise,  the  targets  are 
real,  so  there  are  no  verification  issues  associated 
with  those  systems.  In  many  cases,  the  threat 
systems  can  be  used  to  represent  other  threats 
that  are  very  similar.  In  these  cases,  the  term 
“surrogate”  will  be  used.  The  only  thing  that  must 
be  verified  when  using  “surrogate”  objects  is  that 
the  threat  data  is  a  reasonable  surrogate  for  the 
desired  system.  Lastly,  the  environments  are 
inherently  provided  in  the  data.  The  environments 
are  not  easy  to  control  or  manipulate,  however. 
From  a  data  standpoint,  the  user  is  pretty  much 
limited  to  the  conditions  that  existed  on  the  day  of 
the  flight  test.  From  a  signal  to  noise  standpoint, 
however,  noise  can  always  be  added  to  represent 
a  degraded  environment.  This  process  cannot  be 
performed  in  the  other  direction,  however.  In 
other  words,  there  is  little  that  can  be  done  to 
improve  individual  data  which  was  taken  under 
poor  conditions  (this  assumes  that  normal 
processing,  such  as  integration,  has  been 
performed  already  to  make  the  data  as  “clean”  as 
possible). 

Virtual  Flight  Testing  can  be  used  as  both  an 
enhancement  to  traditional  simulations,  as  well  as 
a  verification  tool.  The  virtual  flight  test  data  can 
be  put  in  identical  form  as  the  simulated  data  to 
perform  side  by  side  comparisons  for  verification. 
This  is  a  powerful  approach,  challenging  the 
verification  team  to  identify  the  “live”  from  the 
simulated  data  set. 
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Process 


The  virtual  flight  testing  process  starts  with 
a  large  database  of  live  flight  test  data,  as  one 
might  imagine.  In  general,  the  characteristics  of 
the  data  collection  sensor  are  well  understood. 
For  example,  the  sensitivity,  noise  levels,  scan 
and  beamshape  losses  and  pulse  compression 
gain  are  generally  well  characterized.  This 
allows  the  data  to  be  manipulated  with  these 
factors  being  fully  compensated  for.  An 
example  would  be  if  data  were  taken  with  a 
phased  array  radar  on  an  object  at  a  low  scan 
angle  (with  associated  scan  losses),  this  same 
data  could  be  made  to  represent  a  higher  scan 
angle  by  adding  the  appropriate  losses  in  the 
form  of  added  noise.  Since  the  loss  as  a 
function  of  scan  angle  is  known,  this  can  be 
done  very  easily.  Another  example  is  that  the 
pulse  compression  effects  can  be  “taken  out”  by 
performing  an  inverse  fast  fourier  transform 
(IFFT).  This  will  bring  the  data  back  to  its 
original  In-phase  and  Quadrature  (I  &  Q)  form, 
which  is  commonly  used  in  simulations.  This 
technique  has  also  been  demonstrated  on  an 
actual  waveform  generator  program,  which  used 
the  “decompressed”  I  &  Q  data  to  generate  a 
real-time,  Pseudo-Random  Noise  (PRN)  coded 
waveform. 

Next,  an  alignment  and  compensation  of 
the  signature  data  must  be  performed.  The 
process  uses  the  metric  trajectory  data  to 
properly  align  and  phase  compensate  the 
signature  data.  This  essentially  puts  the 
signature  at  the  desired  location  and  time  on  the 
trajectory  and  gives  it  the  correct  phase.  This  is 
especially  important  if  the  data  is  to  be  taken 
back  to  its  I  &  Q  format  as  referenced  earlier. 

A  data  gap  filling  process  may  be  required 
next,  if  all  the  data  is  not  at  the  desired  sampling 
rate.  This  can  be  performed  to  gain  a  factor  of 
two  (or  so)  in  data  rate,  without  corrupting  the 
results.  This  process  is  basically  an 
interpolation  process.  For  example,  if  the  data  is 
taken  at  1 ,000  Hz  (pulses  per  second),  but  a 
2,000  Hz  rate  is  desired,  every  second  pulse 
can  be  generated  by  interpolating  between  the 
surrounding  existing  pulses.  This  will  fill  in  the 
data  to  the  desired  data  rate.  A  similar  process 
can  be  done  to  filter  the  data.  If  a  250  Hz  data 
rate  is  desired  in  the  example  above,  only  every 
fourth  pulse  would  be  used.  This  will  “thin”  the 
data  to  the  desired  data  rate. 


We  often  wish  to  isolate  the  target  signature 
from  the  surrounding  “noise  gates.”  This  is  a 
simple  process  that  involves  removing  all  the 
range  gates,  except  those  of  interest,  typically 
containing  the  target  or  other  objects  of  interest. 
At  this  point,  a  new  signature  file  is  created, 
containing  only  the  objects  of  interest  with  their 
inherent  motion  about  their  center  of  gravity.  This 
aspect  is  important,  because  the  object  can  now 
be  flown  on  “new”  three  DOF  trajectories,  so  long 
as  the  viewing  geometry  constraints  are  not 
violated. 

For  combined  simulated  and  virtual  flight 
testing,  the  same  process  is  used  to  isolate  the 
target  signatures.  In  addition  to  adding  new 
trajectories,  simulated  objects  (signatures  and 
trajectories)  can  now  be  superimposed  into  the 
scene.  This  gives  us  the  capability  to  add  any 
simulated  objects  we  desire  to  the  virtual  flight 
test.  This  is  extremely  important  when  we  wish  to 
characterize  performance  against  unknown, 
postulated  or  follow-on  threat  systems,  which  we 
have  no  data  on. 

The  process  described  above  offers  many 
opportunities  to  perform  a  variety  of  testing 
applications.  We  can  test  algorithms  against  live 
data  and  vary  the  signal  to  noise  ratios  (SNRs). 
Fortunately,  most  of  the  data  is  collected  at  high 
SNR  and  noise  can  be  added  to  replicate  the 
effects  of  lower  sensitivities,  as  would  be 
experienced  at  longer  ranges  or  under  heavier 
loading  scenarios.  Similarly,  most  of  the  data  is 
collected  at  high  data  rates  and  the  user  can  fill  in 
or  thin  the  data  to  replicate  different  data  rates  as 
desired.  This  is  also  a  critical  algorithm  test 
parameter,  used  to  establish  how  much  data  is 
required  for  the  algorithm  to  converge  upon  a 
solution.  Different  geometries  can  be  generated 
(assuming  that  the  range  of  aspect  angles  is 
available  in  the  data).  Loading  excursions  can  be 
performed  by  merely  replicating  trajectories  (again 
assuming  that  no  geometry  constraints  are 
violated). 

Example 

Figures  1 , 2  and  3  give  an  example  of  a 
virtual  flight  test.  The  test  scenario  is  a  Theater 
Missile  Defense  intercept  with  a  kinetic  kill 
interceptor.  Due  to  the  extreme  lack  of  relevant 
flight  test  data  of  this  type,  this  is  a  natural 
application  of  virtual  flight  testing.  There  are 
several  “surrogate”  objects  used  in  this  test,  again 
due  to  the  lack  of  real  TMD  kinetic  kill  intercepts. 
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Some  examples  of  the  surrogates  are  that  a 
Chinese  Naval  RV  was  used  to  represent  a 
surviving  separated  warhead.  The  RV  has 
suitable  size,  mass  and  signature  to  represent 
the  desired  warhead  in  this  scenario.  Various 
strategic  and  domestic  fragments  and  closely 
spaced  objects  were  used  to  represent  small 
and  large  post  intercept  debris.  Again,  the 
objects  have  the  correct  sizes  and  signatures  to 
represent  the  desired  pieces  of  debris.  Figure  1 
shows  the  isolated  signatures  with  their  inherent 
motion  about  their  centers  of  gravity.  These 
plots  show  range  along  the  abscissa  and  time 
along  the  ordinate.  The  colors  represent  the 
RCS  intensity,  according  to  the  scale  shown. 
Notice  that  the  RV  can  be  seen  to  tumble  in  this 
range,  time  intensity  plot.  The  signature  of  the 
RV  can  also  be  seen  to  increase  significantly 
when  the  aspect  angle  approaches  broadside 
(when  the  width  of  the  signature  is  at  its 
narrowest).  As  an  example  of  the  flexibility,  if  a 
doubling  of  the  tumble  rate  were  desired  for  this 
signature,  the  time  scale  would  simply  be 
halved. 

Figure  2  shows  the  virtual  intercept.  The 
target  is  range  aligned  at  0  meters.  The 
interceptor  can  barely  be  seen  to  enter  the 
range  window  at  101 .7  seconds.  The  intercept 
event  is  clearly  visible  at  102  seconds,  with  the 
corresponding  debris  patterns  from  both  the 
target  and  the  interceptor.  The  resulting  target 
debris  pattern  has  several  larger  objects 
embedded  in  it,  including  a  surviving  RV.  This 
can  be  further  seen  in  the  “Zoom”  of  Figure  3. 


This  virtual  flight  test  was  specifically  designed  to 
test  algorithms  for  kill  assessment,  against  the 
stressing  case  involving  an  unsuccessful  intercept 
event.  Several  other  large  objects  are  also 
embedded,  including  the  rocket  nozzle  and  a 
couple  of  large  tank  segments. 

The  resulting  debris  patterns  can  easily  be 
modified  to  represent  more  or  less  dense  debris 
clouds,  by  simply  increasing  or  decreasing  the 
number  of  objects  in  each.  Likewise,  the 
expansion  velocities  (how  quickly  the  debris 
spreads  out  in  range)  can  be  made  faster  or 
slower  by  changing  the  range  or  time  scales.  The 
impact  point  can  be  changed  by  simply  realigning 
the  interceptor  and  target.  Additionally,  new 
objects  can  be  embedded  in  the  debris  to 
represent  other  types  of  segments  or  associated 
hardware. 

As  an  upgrade  to  this  capability,  XonTech  is 
currently  integrating  the  U.S.  Army’s  Kinetic 
Impact  Debris  Distribution  (KIDD)  model  for  virtual 
flight  testing.  KIDD  will  give  the  “approved”  debris 
distributions  and  associated  velocities  for  a  variety 
of  warhead  types  and  intercept  conditions.  This 
will  allow  parametrics  to  be  performed  over 
various  lethality  conditions.  The  virtual  flight 
tester  will  then  create  the  resulting  environments 
with  the  applicable  real  data  to  fit  the  debris 
density,  expansion  velocity  and  lethality  conditions 
specified. 
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Figure  2.  Virtual  Flight  Testing  Example  -  Synthetic  Hit  to  Kill  TMD  Intercept  for  KA  Analysis 
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Figure  3.  RTI  of  Virtual  TMD  Intercept  with  Surviving  RV  (Zoom) 
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Possibilities 


XonTech  has  a  database  of  over  20,000 
different  flight  test  objects,  collected  on  over 
1 ,400  different  flight  tests.  It  is  this  extensive 
database  of  re-entry  vehicles,  associated 
objects,  tanks,  guidance  and  control  sections, 
fragments,  shrouds,  calibrations  spheres,  etc. 
that  makes  the  concept  of  virtual  flight  testing 
possible.  Additionally,  this  data  was  collected  by 
a  variety  of  different  sensors,  so  virtual  flight 
testing  is  possible  at  sensor  frequencies  ranging 
from  UHF  to  millimeter  wave  (MMW). 

A  host  of  different  applications  have  been 
envisioned  for  virtual  flight  testing.  Some 
scenarios  of  interest  to  the  Ballistic  Missile 
Defense  community  are  discussed  in  the 
following  list.  There  is  no  implication  of 
“approved  or  design-to  threat”  for  any  U.S. 
defense  system  intended  by  this  shopping  list.  It 
is  merely  a  list  of  possible  scenarios  and 
applications  which  could  be  “virtual  flight  tested.” 

1 )  Create  “clouds”  of  closely 
spaced  objects  with  fragments,  associated 
objects  and  deployment  hardware  to  test 
loading  and  discrimination. 

2)  Add  calibration  spheres  as 
“traffic  decoys”  to  test  bulk  filtering 
algorithms. 

3)  Embed  re-entry  vehicles  in  chaff 
clouds  (live  or  simulated). 

4)  Combine  tumbling  replicas  with 
tumbling  re-entry  vehicles  to  test 
discrimination  algorithms. 

5)  Create  a  fragmenting  or 
segmenting  booster  to  test  loading,  split 
track  and  discrimination. 

6)  Simulate  an  In-flight  failure  of  a 
Post  Boost  Vehicle  or  Tank. 

7)  Generate  a  discrimination 
database,  using  virtual  flight  testing. 


8)  Generate  a  kill  assessment 
database,  using  KIDD  and  virtual  flight 
testing. 

9)  Combine  replicas  and  re-entry 
vehicles  in  a  common  scenario  to  test 
advanced  discrimination  algorithms. 

1 0)  Create  a  catalog  of  “surrogate 
objects”  suitable  for  community  use  and 
testing  of  algorithms. 

This  shopping  list  provides  only  a  small 
sample  of  the  possibilities  offered  by  virtual  flight 
testing.  It  is  intended  to  provoke  the  reader  to 
come  up  with  his  or  her  own  applications  and 
ideas  for  virtual  flight  testing. 

Summary 

Virtual  Flight  Testing  offers  a  world  of 
diversity,  flexibility  and  possibility  to  the  test  and 
verification  community.  The  cost  savings  are 
tremendous,  while  overcoming  many  of  the 
limitations  associated  with  traditional  digital 
simulations.  Virtual  flight  testing  allows  the 
tester  to  use  REAL  data,  on  REAL  objects,  with 
REAL  signatures,  taken  by  REAL  sensors  in 
REAL  environments.  This  overcomes  many  of 
the  verification  and  validation  issues  facing  the 
test  and  simulation  community  today. 

One  example  of  a  virtual  flight  test  was 
presented,  showing  the  benefits  and  flexibility  of 
the  process.  A  variety  of  other  possible  tests 
and  applications  were  presented  to  give  the 
reader  a  better  appreciation  as  to  how  many 
different  types  of  scenarios  and  tests  can  be 
performed. 

In  today’s  environment  of  limited  budgets 
and  expensive  flight  tests,  virtual  flight  testing 
offers  a  credible,  proven  approach  to  reduce 
cost  while  maintaining  high  levels  of  testing 
confidence. 
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